Rayleigh triangles and non-matrix interpolation 
of matrix beta-integrals 

Yu. A. Neretin 1 

A Rayleigh triangle of size n is a collection of n(n + 1) /2 real numbers 
At;, where l^l^k^n; these numbers decrease with growth of k for fixed I 
and increase with growth of k for fixed k — l. We construct a family of beta- 
integrals over the space of Rayleigh triangles; these integrals interpolate 
the matrix beta-integrals of the Siegel, Hua Loo Keng and Gindikin types 
with respect to the dimension of the basic field (R, C or the quaternions 
H). We also interpolate the Hua-Pickrell measures on the inverse limits 
of the symmetric spaces U(n), U(n)/0(n), U(2n)/Sp(n). 

Our family of integrals also includes the Selberg integral. 

0.1. Interpolation of homogeneous spaces. It is well known that the 
representation theory of semisimple groups admits a partial interpolation on the 
level of special functions. The most old (and the most simple) construction of 
this kind is the Hankel transform pQ, Let / be a function on R n depending 
only on r = (x\ + • ■ ■ + a;^) 1 / 2 . Consider its Fourier transform 



Averaging e lax over spheres of radius p — (a 2 + ■ ■ • -l-a 2 ) 1 / 2 , we obtain that f(p) 
is given by 

/>oo 

/O) - H n f(p) := I J n ,2-x{rp)f{r)r n/2 dr, (0.1) 
where J v {z) is a Bessel function 
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Obviously, the operator H n is a unitary operator in the space L 2 (R + , r" _1 dr) 
and satisfies to the property if 2 = 1. 

By our construction, the number n is integer, since n is the dimension of 
the space R™. Nevertheless the Hankel transform 2 (0.1) is well defined for all 
complex n satisfying the condition Ren > 0. By the Hankel theorem, for all n 
the identity H 2 = 1 holds; moreover for real n > the operator H n is unitary 
in the space L 2 (R+, r n-1 <ir). 

Similarly, the index hypergeometric transform for some exceptional values 
of its parameters gives the spherical transforms for all the hyperbolic symmetric 
spaces (|3], |U, 0). For semisimple groups of rang > 1 the interpolation can 
be performed only for functions depending on eigenvalues or singular values of 

1 Supported in part by NWO Grant 047-008-009 

2 We use a nonstandard notation for the Hankel transform. 
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matrices. Various topics related to this interpolation are discussed UJ, 0, [E]> 

0.2. Purposes of the paper. This paper has two aims: 

- to construct an interpolation of the Hua-Pickrell measures ([H], El> 
|14p on the inverse limits of symmetric spaces. 

- to construct an interpolation for matrix B-functions (see |15j . in 
this case we intend to replace the dimension of the ground field dimK, where 
K = R, C or quaternionic algebra H, by an arbitrary complex number. 

Recall that the second problem is not standard from the point of view of the 
modern interpolation ideology, since the matrix _B-function is not a function of 
matrix eigenvalues. The both goals are achieved only partially. Namely this is 
done for simple groups of the series A. 

Emphasis that our family of beta-integrals includes the Selberg integral. 

0.3. Hua Pickrell measures and their interpolation. Now we formu- 
late in more details the problem about the Hua Pickrell measures. Below we 
do not return to this topic and the rest of our text is a 'play in integrals'. 

By U(n) we denote the group of unitary matrices of order n. Consider the 
map T : U(n + 1) — > U(n), given by 



T 



P R f) = T-R(1 + P)~ 1 Q, 



P Q 



where € U(ra+ 1) is a block matrix of size (1 + n) x (1 + n). It can be 

easily checked that the image of the Haar measure on U(n + 1) under the map 
T is the Haar measure on U(n). Hence we can construct the inverse limit it of 
the chain 

■ • • U(n - 1) \J(n) U(n + 1) . . . . (0.2) 
Consider a sequence of matrices g n € U(n) satisfying the condition 

9n = T(.9„+i) 

(the set of such chains is in an one-to-one correspondence with the space it). 
Denote by 

^nii ■ ■ • j Vrin the eigenvalues of the matrix g n . 

Thus, for each element of the space il and each n = 1, 2, . . . we assign the 
collection of points v n ii • • ■ ? v nn of the circle \v\ = 1. 

In other words, we obtain a stochastic process with a discrete time, its value 
in a moment n is an n-point subset of the circle. 

The construction admits three following variations, see ■ 

First, we can replace the Haar measure dg U(n) by the two-parameter family 
of the Hua-Pickrell measures 

Cn(a) det(l + g)' T det(l +g) w dg, 

where a S C is fixed and the normalizing factor c„(cr) is defined by the condition: 
the measure of the whole group U(n) is 1. These measures form a consistent 



2 



system with respect to the maps T, and we again can consider the projective 
limit of our chain. 

Second, the Hua-Pickrell measures can be replaced by a family of measures 

depending on an infinite family of complex parameters <7i, 02, In this case, 

the measure on the group U(n) is given by 

Cn((ri, 0-2, ■ ■ ■ ) det(l + g) a " det(l + gf" x 

n—1 n—1 

x [] dct(l + [g]^-^ J] det ( : + l9hp- Wj+1 dg, 
j=i j'=i 

where [g]j denotes the left upper j x j block of the matrix g. 

Thirdly, this construction survives for all the classical compact symmetric 
spaces. Consider the inverse limits il/D and il/Sp of two chains 

. . . < — U(n - l)/0(n - 1) < — U(n)/0(ra) < — ... (0.3) 
. . . < — U(2n - 2)/Sp(2n - 2) < — U(2n)/Sp(2n) < — ... (0.4) 

Apparently, the three spaces il, ii/D, H/Sp themselves can not be interpo- 
lated. However our basic construction (Theorem 3.1) can be interpreted in the 
following way. We construct a large class of Markov stochastic processes, whose 
value in a moment n is an n-point subset on the circle. In particular, this class 
contains processes on the circle related to the chains (0.2) - (0.4). 

I am grateful to G.I.Olshanski for discussion of this subject. 
§ 1. Main results 

1.1. Rayleigh triangles. Let A be an Hermitian n x n matrix, let B be its 
left upper (n — 1) x (n — 1) block. Denote by ^i^^2^ ■ ■ ■ ^fJ-n the eigenvalues 
of the matrix A and by Ai^A2^ . . . ^A„_i the eigenvalues of the matrix B. 

By the Rayleigh theorem (see for instance |TJ|, [TH)h 

/ii<Ai</Lt2<A 2 <M3< ■ ■ • <A„_i^„. (1.1) 

We define a Rayleigh triangle L of size n as a collection of n(n + l)/2 real 
numbers 

An 

A21 A22 
A31 A32 A33 ,j 2^ 

Aril A„2 A„3 . . . A„(„_ 2 ) A n ( Tl _i) A„ n , 

satisfying the system of inequalities 

■ ■ • ^A(j +1 )fc<Aj/s<A(j + i)(fc + i)< . . . (1.3) 
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for all the possible j, k. In other words, the numbers Xj a decrease south-west- 
ward direction and increase south-east-ward. 

We denote the space of all the Rayleigh triangles of the size n by Ji n - By 
%, [a, b] we denote the set of all the Rayleigh triangles of size n satisfying the 
condition 

(hence all the numbers Xj a lie between a and b). 

It is known, that some matrix integrals can be reduced to the integration over 
the spaces [a, b] ; at least this was a way of the evaluation of spherical functions 
of the group GL(n, C) in Gelfand's and Naimark's book [TQ. A.Okounkov and 
G.I.Olshanski [201 obtained a representation of the Jack polynomials as some 
integrals over the space of Rayleigh triangles. This work was a standpoint for 
the present paper; see also Kazarnovsky-Krol's work |21| . 

1.2. Natural measures on % n . By K we denote R, C or the quaternionic 
algebra EL Assume 

9 = dimK/2. 

Let us define a natural measure {dx} on K: for the real numbers it is the 
Lebesgue measure, for the complex numbers x — u + iv we put {d(u + iv)} = 
dudv, and for the quaternions x = u+iv+jw + kz we give {d(u + iv +jw + kz} = 
du dv dw dz. 

Consider the space Herman) of all the Hermitian matrices 3 of size n x n 
over K. Equip this space by the Lebesgue measure 

l[{dt k i}l[dt 

kk- 

k>l k 

For a matrix T denote by [T]j its left upper block of size j X j. 
For each block [T]j of the matrix T denote by 

Xji^Xj2^i ■ ■ ■ ^Xjj 

the eigenvalues of the matrix [T]j. Thus we obtain the map 

HermK(n) — » 3?„(— oo, oo). 

Proposition 1.1. 4 The image of the Lebesgue measure on HerniK(n) under 
the map Herman) — > 3l n is 

C n (9)v e (L)dL, (1.4) 

3 A square matrix Z over K is Hermitian if its matrix elements z a( 3 satisfy the condition 

z af3 = ~Z(ia- 

4 The author does not know is this statement published somewhere; it can be observed from 
the calculation |19l . §11.9. It is known in a folklore (since the integral representation of the 
Jack multivariate hypergeometric functions 1201 . 1211 ideologically depend on this statement. 
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where 



r <>(£) = n fr ~ a x 



n n \\j-i)a - ^j P \ e 1 
n n (A,-/? - a, q ) 5 



x n ( a «9- a «p)' (!- 5 ) 

here d£ zs the Lebesgue measure on the space of Rayleigh triangles 

dL = n n d v 

and C n {9) is the normalizing constant 

„-n(n-l)0 

C„(0) 



r n(n-l)/2(g) ' 



By MatK(«, m) we denote the space of matrices of size nxm over K. Assume 
n<m, i.e., the number of rows does not exceed the number of columns. By {T}j 
we denote the matrix consisting of the first j rows of the matrix T. For each 
j = 1,2,..., n, we denote by 



the eigenvalues of the matrix {T}j{T}*, obviously all these numbers are non- 
negative. Thus we obtain the map 

Mat K (n,m) -> R„[0,oo). (1.6) 

PROPOSITION 1.2. T/ie image of the Lebesgue measure on Mat(n,m) under 
the map (1.6) is 

C nm (6)TP(/l)ze(L)dL, 
where t$(L) is given by (1.5), 



#c) = n A $r n+i)9 " 1 ' 
P =i 



and 



T n(n-l) {6)I] n^ r{{m _ j + 1)e y 



1.3. Ultra-beta-integrals. Wc deduce two following integrals (1.7)— (1.8) 
over the space of Rayleigh triangles. 
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Fix complex numbers aj, Tj, where j — 1, ...,n, and also the complex 
numbers 9j a , where j, a range in the domain l^a^j^n — 1. Then 

n— 1 j ^Oj-aj + i-8j a n \0n — 1 

TT TT j a tt A n P 

11 11 (1 + A, a )^+i+^ 11 (1 + A np )-« 

K»[0,oo) J = lQ=1 P =1 

n-1 II l^'a - ^(j + l)p| 03 ° 1 

7=1 11 1 • ?/3 JQ '' l<)j< <n 

= n n^)-n rfa)r(Tj "VF Jl% " 1)J - 

l<a<jXn-l j'=l ^ Tj ^ 

The conditions of absolute convergence of this integral are 

j'-i 

Re0,- Q > 0, Reo-j > 0, Rer^ > Recr.,- + Re0y_i) a . 

Q = l 

As above, let Tj, Oj, 9j a be complex numbers. Then 

n— l i 

nn 

j=l a=l 



/n— 1 j 
A 1 ^ 1 



^(-00,00) 

n 

p=l 

n-1 II \^ja - ^{j+l)p\ ia ~ 

= 11 r (M-ll n^wW) ' 

(1.8) 

where i is the imaginary unit. The conditions of the absolute convergence of 
this integral is 

i-i 

Re 9 ja > 0; Re{o- 3 + Tj - 1 - ^ > 0. 

In §3 we derive three more integrals (3.1)-(3.3) in the same spirit. 

1.4. Matrix integrals. Ultra-beta- integrals include many various matrix 
beta- and gamma- integrals of Whishart-Ingham [22], Siegel [2H|, Hua Loo Keng 
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|231 and Gindikin types. 5 We present two series of such integrals now 
and some more series in .4.3. I have not seen some of these integrals in the 
literature. Apparently, varying symmetry conditions on matrices it is possible 
to obtain many other integrals in the same style. 

Consider the space Posk(^) of Hermitian positive definite matrices T of size 
n x n over field K = R, C, H; as above, let 6 = | dimK. Fix complex numbers 
or, . . . , a n and n, . . . , r n . The beta-integrals of Gindikin (see JSJi see a l so FK, 
they extend the Siegel integrals [231 ) have the form 

-Uj det Tj - Tj+1+e [l + T} 3 det r "(l + T) 

Pos K (n) 5 



7r n(«-l)/2 -Q 



r(a,)r(r, - a s - (j - 1)0) 



Let us explain how to reduce them to our integrals. In notation of Subsection 
1.4, the measure dT can be rewritten in the form (1.4), the integrand transforms 
to 

n % "j -o-j + i -9 n \CT n — 1 

TT -2- TT — 

11 fl + Ai„) r i- T *+»+ e 11 (1 



A (1 + A ia )^+i+» ^ (1 + A np )^ 



and we obtain a partial case of the integral (1.7). 
Similarly, 

/n— l 
JJ det(l + i[T] i )"^ +CT3 ' +1_e det(l - i[T] i ) _T3 ' +T '* +1_e x 

Hcrm„(K) J = 1 

x det(l + iT)~ an det(l - iTy T "dT = 

_ _n(n+l)/2 9 2n-S ^ TT r ( T J + ~ 1 ~ (j ~ 1 ) 6> ) 

For the field K = C this integral is evaluated in (2H| , Q2| i one of partial cases 
of this integral is the Hua Loo Keng integral [23] 



J dct(l +T 2 )- a dT 1 



Herm„ (K) 

it corresponds to 



cr - (n - j) 



5 The integrals from author's work 1161 are not absorbed by ultra-beta-integrals. 
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Further, 



Mat„ 



det(l + {T}j{T}*) T j- T j+ 1+e det(l + TT*)^ 



mn9 TT £(gj ) r ( T j ~ g j ~ (j ~ 

1 = 1 rfajrtfm-j + i)*) ' 



This series of integrals includes the Hua Loo Keng integrals |2"1] 

detfl + T 2 )- T dT. 



I 



Mat m ,„(K) 

Some more series of matrix integrals are discussed below in 4.3. 

1.5. Projective systems of measures. Fix an infinite sequence of com- 
plex numbers ci, o~%, Assume Tj — Wj. Fix also reals numbers 9j a , where j, 

a range in l^a^j. Let these numbers satisfy the inequalities 

i-l 

6 ja > 0; 2RCCTJ > 1 + ^ {j _ x)a . 

a=l 

Further, fix n. Denote by ^„(X) the integrand in (1.8). Consider the prob- 
ability measure 

X n = C n ■ ?fin(&)d£j 

on Jt n (— oo, oo), where c„ is a normalizing constant (we choice it to make the 
measure probability) . 
Consider the map 

T : 3l n+1 (-oo, oo) — > 3? n (-oo, oo), 

that erases the last row of a Rayleigh triangle L E 3i n+ i(— oo, oo). 

Theorem 1.3. The image of the measure >?„+i under the map T coincides 
with >c n . 

Hence, by the Kolmogorov theorem on the inverse limits, there exists a 
canonical probabilitic measure on the space of infinite Rayleigh triangles 
such that its projection to 3?„(— oo,oo) coincides with x n for all n. 

Possibly, this construction is too general. We mention its definitely reason- 
able partial case. Assume 9j a = 9 for all j, a and Oj = a+j9. Then we obtain a 
series of measures interpolating the Hua-Pickrell measures on the inverse limits 
of symmetric spaces 

lim U(n)/0(n); lim U(ra); lim U(2n)/Sp(n); 

oc< — n oo< — n oo< — n 

these cases correspond to 9 = 1/2, 1, 2. With comparison with 0.3, we change 
the notation, namely Xj a is connected with Vj a from 1.3 by the formula Xj a — 
i(l-u ja )/(l + u ja ). 
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Remark. The space 3?„[— oo, oo] discussed in this subsection imitates the 
space of Hermitian matrices, but in Subsection 0.3 we told about unitary ma- 
trices. The passage from unitary matrices to Hermitian matrices is given by the 
Cayley transform 

T=-l + (l + g)- 1 ; fireU(n), T S Herm c (n) 

For a matrix T, we denote by (T)j its lower right block of size j x j. Then (see 

El) 

det(l + [g] k ) = 2- n+k det(l + (T) n _ fc ) • det(l + T)" 1 

and this allows to reduce the measures from 0.3 to the form c n ■ ^p„(£)<iC. 

1.6. Trapezoids. Integrals (1.7)-(1.8), and also integrals (3.1)-(3.3) given 
below allow a minor extension. For definiteness, we discuss the integral (1.7). 
Name by a Rayleigh trapezoid a collection of numbers 



A m2 ... A„ 



A( m+ l)l A( m+1 ) 2 . . . A (m + i )(m + 1 ) 

A ra l A„ 2 A„ 3 . . . ^ra(ra-l) A„„, 

satisfying the interchanging condition (1.3). We denote the set of all such trape- 
zoids by ft™. 

The following formula holds 



n— 1 j yVj-Vj+i-Uja n \fra-l 

-i r "rip 



fifl f 1 + V)" + 1 + ^ ' n (1 + X np )rn 

) 34 X 

n-1 

n n i^ja - A(j + i)pi^ a_i 



X 



n-1 

n n (Aj/9 - A ia )<^+^- 2 i<p<«<» 

1 r((j + l)x)r((T m + jx)r(r m - <r m - (m + j - l)x) 



n 



._, r(x)r(r m -jx) 

x n r ^-) n r ^ )r ^-^-p^^-i) a ) . (1 . 9) 
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Emphasis, that for m = n we obtain the Selberg integral (see [2*3 > EE3]) : 



0< M1 <-<M^ =1 (1+MP)T 1^<^ 



A T(a + g(j - l))r(r - £- g(n - j + i))r(jg) 

1 = 1 r(r-(9(j-i)r(fl) 1 ■ j 

(we denote the variables X np by and >c by 0; the remaining parameters for 
m = n are lacked). 

1.7. Conjecture. Our ultra-beta-integrals are some kind of a superstruc- 
ture over the Selberg integral. The Selberg integral is the simplest representative 
of multidimensional beta-integrals. It seems likely that a superstructure of this 
type exists for other beta-integrals related to root systems of the series A n , for 
instance for the weight function of the Macdonald polynomials. Anyway, for 
the Macdonald polynomials are possible integral representations similar to |2(J|. 
EI], see I2HI- 

1.8. Further structure of the paper. Our §2 contains main lemmas. 
Using them, in §3, we easily deduce the projectivity theorem, the integrals (1.7)- 
(1.8), (1.9), (3.1)-(3.3). In §4, Propositions 1.1-1.2 are proved, in this section 
we discuss some more matrix integrals. 

§2. Change of variables 

We use changes of variables of the Anderson type (2H| • 
2.1. Basic lemmas. Fix real numbers a < b; we allow a = — oo or b — oo. 
Let fixed numbers Ai, ... A„_i satisfy the inequalities 

a < Ai < • • • < A n _i < b. (2.1) 

Denote by S[a, 6; A] the set of all collections /Ui, . . . , fi n , satisfying the inequality 

a < /Ltj < Ai < fi2 < ■ ■ ■ < A n _i < /i„ < b. (2.2) 
Lemma 2.1. Let complex numbers a, r 6±,. . ., 9 n ~i satisfy the conditions 

Re6> Q >0, Rc(7>0; Re r > Re a - V] Re a . 

a 

Then 

f - fi a ^ 1 

/ n d+ u \r - n lAa-^i^ -1 n o^-m n d^ p = 

r(r-a-E^) n 

= m x 
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Lemma 2.2. Let complex numbers a, r, 9\,. . . ,8 n -i satisfy the conditions 
Re9 a >0; Re(a + t - Rc 9 a - 1) > 0. 



Then 



/n 
\\{l + ll i p )- a (l-i^)- T ■ \{ \X a -ii P \ K ~^ 

E[-oo,oo;A] P=1 l<a<n-l,l^n 

x n - n d ^p = 

7T2 2 -— r(<7 + r-£0 Q -i) n r (^) 



l^a<n-l 
X 



r(r)r(a) 

X I] (A|8 - Aa)^"" 1 ■ IJ(1 +iAa)-'- ^(l-iAa)"^-. (2.4) 

l<ce</3<n-l l<a 

2.2. Variants. There are three following variants of Lemma 2.1-2.2. 

A) Let Recr > 0, Re(9 Q > 0, ReV> > 0. Then 

/n 

H[0,oo;A] p=1 K«^-M^r. 

X 11^9- Mp) ]J d ^P = 

l^p<g^n l^p^n 

n-1 

= 9 »r((r) [J r(0 a ) Y[ a ct - 1+0 ° e -^ A ° x 

a — 1 l^aSCn — 1 

x n (A/3 - X a ) e ' x+e ' , ~ 1 - (2.5) 

l^a</3^n-l 

B) Let Rc?/> > 0, Re 9 a > 0. Then 

r. n 

J U e ^ 2p n iao-mpi 6 ^ 1 n (^-mp) n ^= 

H[_oo,oo;A] P=1 l^p^n l^p<q^n l^p^n 

, n— 1 

= v^ C xp{-| ^ a «}^" /2 ^ 9a n n ^ - Aa)'--*"- 1 - 

a a=l l^a</3^n-l 

(2.6) 
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C) Let -oo < a < b < oo. Then 

« n 
2[a,b;\] P=1 

x n iAa-Mpi 9a_i n (vq-^p) n 

n-1 

i»r(r) n r((? a ) 

xn(Aa-ar +e "- 1 (6-A a r +9 »- 1 I] (A /3 -A a ) e °+^- 1 . (2.7) 

a=l l^a</3^n-l 

2.3. Change of variables. Let the parameters Ai < • • • < A„_i be fixed, 
and let the variables fii, . . . , fi n satisfy the interchanging conditions 

Hi < Ai < fi 2 < • • • < A„_i < fin- (2-8) 
Introduce the new variables £i, . . . , ?? by the formulas 

n (mp - A Q ) 

fc - - T (2 - 9) 

»?= E ^- E v ( 2 - 10 ) 

Lemma 2.3. a) The map (fn, . . . ,fi n ) . . . ,^ n _i,?j) given by formula 

(2.9)-(2.10) is a bijection of the domain (2.8) to the domain 

£i>0, 6>0, £„_i >0, ryeM. (2.11) 

b) 7/ m addition Ai > 7 £/ien i/ie inequalities 

< /Ui < Ai < ^ 2 < A 2 < • • • < A n _i < m« (2.12) 

are equivalent to the inequalities 



n-l 



a=l 



£i>0, 6>0, £„-i>0, *7>ElK ( 2 ' 13 ) 



c) T/ie Jacobian of the change of variables (2.9) - (2.10) is 

l<p<9<n 



II (A/J-Aa) 

l^a</3^n-l 



(2-14) 
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Proof. Consider the function 



R{x) := 



n (x-np) 

1 sCpsCn 

n {x-XpY 

l</3<n-l 



The residues of are 



res 

X — \ a 



Hence 



R(x) = X — T] 



E 



l<a<n-l 



x - A Q 



(2.15) 

(2.16) 
(2.17) 



Positivity of the variables £ Q immediately follows from the inequalities (2.8). 

Conversely, assume that all the £ a are positive. The expansion (2.17) implies 
that the function R(x) is +oo at the left end of the interval (A a ,A a +i), at the 
right end it equals — oo. Hence R(x) has at least one root on this interval. The 
same is valid on the rays (— oo, Ai) and (A n _i, +oo). Hence, for positive £ Q all 
the roots \x p of the equation R(x) = are real and satisfy the inequalities (2.8). 

Thus the statement a) is proved. We know that the interval (— oo, Ai) con- 
tains one root of the equation R(x) = 0. It is positive iff R(0) < 0; this proves 
b). 

To evaluate the Jacobian, we observe 



dfip 



Hence the Jacobian is 



where 



lip X a 

71-1 



drj 
djJLp 



= 1. 



A = det 



fj,! -Xl 
M2-Al 



1 




1 




M1-A2 


Ml 


-A r 
1 


—1 


M2-A2 




-A r 


— 1 


1 




1 




Mn-A2 




-A, 





1\ 

1 



(2.18) 



Lemma 2.4. The following identity holds 



II (Mp-M<?) II (A/3-A a ) 



A = 



l^p<q^n 



n (p-p - A ") 

l^p^n, l^a^n — 1 



Proof. We use the Cauchy determinant 
1 



n inq-ih) n (A/a-A a ) 



det 



l^p<g^n 



lsga</3sgra-l 



A; 



II (Mp - A a ) 



(2.19) 
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Further, we multiply the last column of the determinant by A„ and turn A„ 
to infinity. This proves Lemma 2.4. □ 

It remains to notice that 

n k-a q | 

|?i?2 • ■ -?n-i| = — 



n (a^-a q )2 ' 

l^a</3^n-l 

and this proves Lemma 2.3. □ 

Lemma 2.5. In notation of Lemma 2.3, for each a e C the following identity 
holds 

n—l c 



l^p^n l^a?Cn — 1 a—1 

Proof. Consider a function 



n (x-fip) 

Q(x) = 



(x + a) n ( x - x t3)' 

l</3^n-l 



Its residues are 



res Q(x) = - — — ; res Q(x) = - p — -; res Q(x) = rj + a. 
x=x a A a + a x=-a \_\yAp + a) x=oo 

The sum of residues is 0, this is equivalent to the statement of Lemma. □ 
2.4. Deduction of Lemma 2.1. Denote our integral by I\. Let 

W= [] (A/j-Aa)^ 9 "- 1 . (2.20) 

l^a</3^n-l 

Passing to the variables £ a , rj using Lemma 2.3 2.5, we obtain 



n-l 

X 

c*=l 



where the integration is given over the domain £ Q > 0, rj > ^ 
Further, we change 77 £ Q to the new variables 

n-l 

ESQ ?Q 
— ; «q = 



_ 1 A Q A a (l + A a ) 
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In these variables, 

n— l ^ n—i 

q— 1 a— 1 

and 

n— 1 n— 1 

rj C^ 1 -dr)d£i... d£„_i = JI 1 ' dwdui . . . <2u n _i. 

CK — 1 Q — 1 

The integral converts to the form 



TT 7- — ~~~ — r — s— / -. r— dudvi . . . dv n -x, 

^ (l + \ a y-<>« J (l + u + «! + ••• +v n ^Y 

where the integration is given over the domain u > 0, v a > 0, and we obtain a 
variant of the Dirichlet integral, see the next subsection. 

2.5. Dirichlet integrals. The Dirichlet integrals are the three following 
related integrals 

71+ 1 

n n II r (^) 

/ IT ff"^ 1 - E ^) X " +1 " 1 ^i • • • dt n = J=1 n+1 ; (2.22) 

«!>o („>o,E*„<i 3=1 0=1 T (I2 x j) 



n n n 

n f/^dh ...dt n T( y - £ x,) n r(^; 



/ n / „ (1+E*i)« ^ 

i=i 

n-l x _ 1 
n ' d*l • • • d*n-l 

3=1 

n— 1 n— 1 

ti>0,...,U_i>0;seK (l + «s+ I] t,)«(l-is + J] tj) 2 

3=1 3=1 



(2.23) 



n — 1 n — 1 

2 2 ^7rr(y + z-l- E z,.) J] rfo) 
= ; ; ; ^ — ■ (2-24) 

r(v)r(z) 1 J 

All these integrals can be easily evaluated by successive integration in t±, t 2 , 
. . . , see also Theorem 1.8.6 from |3U| . 

2.6. Deduction of Lemma 2.2. Denote our integral by I2, define the 
quantity W by (2.20). Then by Lemma 2.3, 2.5 we obtain 



T " ± 

^ = rj(i + a a )- CT (i-a a )- T x 

CK — 1 

« n— 1 ^ _ ri— 1 £ _ 7i— 1 
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where the integration is given over the domain (2.11). 

The first and second factors of the integrand are converted to the form 

n— 1 ^ * n— 1 c _ n— 1 ^ n — 1 ^ 

a— 1 LK a— 1 " a—1 " a— 1 " 

Further, introduce the new variables 

v n = — — ; u = V-}_^i 



1 + A|' ' ^1 + A| 



Our integral transforms to the form 

n-l 

Y[(i+i\ a )-° +9 <>(i-i\ a )- T+e <>x 

I 



n-l 

n< 

a=l 

X 



Y[v e a «- l du Y[dv a 



»i>o,...,ti„-i>i),uei 



(i + iu + v a y{i - iu + v 



It remains to apply the Dirichlet integral (2.24). 

2.7. Deduction of integral (2.5). Denote our integral by ^3, define the 
quantity W by formula (2.20). Then 



j n—l ~ ~ _^ n-l n— 1 

# - n ^e-^ /(«7-e^) " ^ n n «■> 



where the integration is given over positive £ Q and 77 > ^ £ a . Introducing the 
new variables 

T-; "a = T-, 

we obtain 





x exp{— ip(u + y^ j v a )}duY\ dv a , 



a_1 M>0,t>i>0,...,D„_l>0 



this comes to a product of r- functions. 
2.8. Derivation of formula (2.6). 

Lemma 2.6. Let X a , /j, p , £ a , n be related by formula (2.9)^(2.10). Then 

n n—l n—l 

E^-E A « + 2 E^ 2 - 

p—l a—1 a—1 
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Proof. Represent the function (2.15) in the form 

«M-«n('-£)n('-£r- 

The coefficient at x~~ x of its Laurent expansion at infinity is 

p<q a<(3 

On the other hand, the sum of residues is zero, i.e., c = —J2£ a - Q 
Denote our integral (2.6) by I4. Passing to the variables £ Q , 77, we obtain 
the trivial integral 

#=exp{-fE^}£ >0r;£R ex p{-^ 2 -V'E^}iie- i ^n^- 

2.9. Another deduction of integrals (2.5)— (2.6). Observe that 

hence the integral (2.5) can be obtained from (2.3) by the substitution 
t = k\ fip — fip/k] X a = \ a /k 

and passing to the limit as k — > 00. Similarly (2.6) can be derived from (2.4). 

2.10. Derivation of formula (2.7). In this case, it is possible to repeat 
literally the calculation from Subsection 2.4 and to reduce our integral to the 
Dirichlet integral (2.23). 

Also, it is possible to change the variables in (2.7) in the following way 

„ b — a ~ b — a 

^p = i -^a T 

Hp — a \ a — a 

Thus we reduce our integral to (2.3). 

2.11. One more integral. Fix constants hi < ••• < Hn+i- Let the 
variables Ai, . . . X n satisfy the inequalities 

Hi < Ai < /j, 2 < A 2 < • • • < A n < Hn+i- (2.25) 

Lemma 2.7. Let Re6i > 0, . . . , ReOi > 0. Then 

/n 
IJ I^-HpI 9 *- 1 II (A/3 — A a ) JJ dX a = 

l^a^n: l^p^n+l l<a</3^n a=l 

n+l 

II r(fl Q ) 

= LI (^-^) 9p+9g " 1 - a= n +1 ■ ( 2 - 2 6) 

iss P «?ssn+i r( x; 6» Q ) 

a=l 
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2.12. One more change of variables. Define the new variables £1,. . . , 
Cn+i by the formula 



II (Aa-/ 



Is 



II ' : /',- /';/)' 



(2.27) 



Lemma 2.8. a) The map A £(A) transforms the set defined by the in- 
equalities (2.8) to the simplex 



n+l 



.^Cp = i; Ci >o,...,c„+i >o. 
P =i 

b) TTie Jacobian of the change of variables 

(Ai,...,A„) i ^ (Ci,...,Cn) 



n (A/3-a q ) 

lsCa</3s;n 



n (mp-m?)' 



Proof. Consider the function 



EI («-Aa) 
//(,:) = ' " " 



n ( x ~ mp) ' 

IsCpsCn+l 



The residues of this function are 

res -ff(x) = C P ; res H(x) = —1. 

The sum of all the residues is 0, hence ( p = 1. Obviously, inequalities (2.8) 
imply positivity of ( p . Conversely, let all the ( p be positive. Taking in account 
the expansion 

^ ^ - Mp 

we observe that the function H(x) has a unique zero on each interval {fi p , /U p +i). 
This proves statement a). 

Further 

dCp _ Cp 
<9A Q A Q — /ip 

Hence the evaluation of the Jacobian is reduced to the Cauchy determinant 
(2.19). □ 
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2.13. Derivation of Lemma 2.7. The change of variables (2.27) converts 
integral (2.26) to the form 



ra+1 



l^p<<j^n+l Ci>0,...,C„>0 p=1 p=1 

Applying C, n +i = 1 — Ci — ■ ■ ■ — (m we obtain the Dirichlet integral (2.22). 
3. Big integrals 

Here we prove Theorem 1.3 on projectivity, and also deduce the ultra-beta- 
integrals (1.7)— (1.8), (1.9), and similar integrals (3.1)-(3.3). 

3.1. Deduction of integrals (1.7)— (1.8). For definiteness, consider the 
integral (1.7). Using Lemma 2.1 we integrate in the variables A„i, . . . , X nn . The 
result is an integral of the same form but over the space 3?„_i [0, oo) of Rayleigh 
triangles of lower size. 

Next, we integrate in A (n _i)i, . . . , A (n _i)( n _i), etc. 

3.2. Projectivity. In the notation of 1.1, let /(£) be a function on 
3^n+i(— oo, oo), depending only on the first n rows of a Rayleigh triangle. To 
derive Theorem 1.3, we must prove that 

J f(Z)dK n+1 (L)= J f{L)dx n (L) 

0i n +i(— oo,oo) — oo,oo) 

But Lemma 2.2 allows to integrate explicitly over the variables A(„ + i)i, . . . , 
A( n _|_i)(„_|_i), After this we obtain the required result. 

3.3. Variants. Formula (2.5)-(2.7) imply the following integrals 



/ II II \?- ai+1 - $ia e-to-+^ x <> ■ J] X^e-^^x 

K„[0,oo) 3 = la=1 P =1 
n-1 II \^ja - \] + l)p\ e3C " 1 

>< n 1<a< J n P X - x-y^e^ x< n (A - - A - } dz = 

= n noj^Un^j ; (3.i) 

l<a<.j<n— 1 j=l 
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f " 1 1 . 1 n 

J Yl cxp{ - - - ip J+1 ) A ja } ' exp{ - - V„ XXp } x 

K n (-oo,oo) J ' =1 " Q=1 p=1 

>< n 1<a<, ifX-A, Q ^^- 2 n<„ (A " 9 _ Anp) dZ = 

n -n/2- ^ 9 u . 1)a 

= (2^)"/ 2 I] T(e ja )ll^ — ; (3.2) 



/n-1 j 
1 [ .[I ~ a) CTi "^ +1 "^ a (b - \j a Y i ~ Ti+1 ~ esa x 

3t n [o,6] 1=1 a=1 

n 

x n( A ™p- a ) <7 "" i ( & - A «p) rn " ix 
P =i 

n-l .11 l^j'a _ A 0+l)pl 3a_1 

x n Ka ^n P X-A, a )^+^^ n ( a «« - A «^ d£ = 

x II r (M-f[ • (3-3) 

l^a^-l j=l Y{aj +Tj+Y. 



3.4. Trapezoids. We start from the integral 



m-i II \^ja - \j+i)p\* 1 

n (A^-A j/3 )^-2 



x TT TT ' Ja > TT A "p x 

II |Aja - A(j + i)p| 9jQ_1 



which is a partial case of integral (1.7) evaluated above. The integration in An 
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is reduced to the evaluation of the integral 

J (An — ^21)* HA22 — Aii)* f_1 dAn. 
A21 

After this integration, we have an integral over 3£^[0, 00) of the form (1.9). Now 
the integration in A21, A22 is reduced to Lemma 2.7. Further, using the same 
Lemma 2.7 we integrate the last integral in A31, A32, A33 etc. As a result, we 
obtain the integral (1.9) over 3?™[0,oo) 

§4. Maps from matrices to Rayleigh triangles 

Here we prove Propositions 1.1-1.2, and reduce some more matrix integrals 
to the ultra-beta-integrals. 

4.1. Proof of Proposition 1.1. Consider the space Herman) of Hermi- 
tian matrices T of size n x n over K = K, C, H. Let us evaluate the distribution 
of the eigenvalues (11, . . . , [i n of the matrix T if the block [T]„_i is fixed. 

Write T as a block ((n - 1) + 1) x ((n - 1) + 1) matrix: 

j, = (T n T 12 \ 
\T 2 i T 22 J 

By this time we fixed Tn; the block Ti 2 ranges in the coordinate space IK™ -1 
with the standard Lebesgue measure; the block T 2 2 ranges in real numbers, at 
last, the block T21 = T* 2 is completely defined by the block 7i 2 . 

Take a unitary matrix g e U(n — 1, K), such that A = gTug^ 1 is a diagonal 
matrix. Consider a matrix 

(Tli T[ 2 \ (g 0\ /Tu T i2 \ (g~ l 0\ ( A 5 Ti 2 \ 
\T2i n 2 J \0 l) \T 21 T 22 J\0 l) \T 21 g- 1 T 22 .) 

We observe that the distribution of the vector T[ 2 coincides with the distribution 
of the vector T12, T 22 = T 22 . 

Thus, we reduce the problem to the following form. Let T has the form 



T = 



/Ai 


. 


Xl \ 





A 2 • 


x 2 





. 


A„_i i n _i 


\Xi 


x 2 . 


■ Xn~l y J 



where x a G K, y € R. Let fj,i,. . . , [i n be its eigenvalues. Denote by {dx} the 
standard Lebesgue measure on the field K (see 1.2). We must find the image of 
the measure {dxi} . . . {dx n _\} dy under the map 

(xi, . . .,X n -l,y) l-> (/XI, . . . ,Hn). (4.1) 
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Define the new variables 

Ca = ka|; V = V- ( 4 - 2 ) 

The image of the measure {dx{\ . . . {dx n -\}dy under the map (4.2), obviously, 
is 

_(n-l)0 n ~ 1 n ~ 1 

T»=m n &- 1 • *i n ( 4 - 3 ) 

v 7 q=1 a=l 

The characteristic equation for the matrix T has the form 

n— 1 ^ 

^-^-E^V = °- ( 4 - 4 ) 

a=l ^ A " 

Since the roots of this equation are fii,. . . , fi n , the left side of (4.4) can be 
rewritten in the form 

II (M-Mp) 



From here, we obtain 



n (m-a^)- 

1</3<ti-1 



n - A a ) 

n (A/? — A a ) ' 



i.e., we obtain the familiar change of the variables (2.9)-(2.10); in fact, this is 
the origin of the formulas (2.9)-(2.10). 

Applying Lemma 2.3 and formula (4.3), we obtain that the distribution of 
the eigenvalues /Ui, . . . , \i n is 

fn-ne n lAip-A Q | 0_1 „ 

fS=ip) n (A,-A Q )^ ' J} ^ - ^ 11 ^' 

l^a</3^n i^p«?^n P-i 

This implies Proposition 1.1. 

4.2. Proof of Proposition 1.2. Consider the space Mat^^-i-fc— l 

(K) of 

n x (n + k — 1) matrices. 

Represent such matrix as a block ((n — 1) + 1) x ((n — 1) + k) matrix T = 
T T \ 

„ . Evaluate distribution of the eigenvalues of the matrix TT* if the 

J21 J22/ 

blocks Tn, T12 are fixed. For this, consider unitary matrices h G U(ra — 1,K), 
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g € U(n — 1 + k), such that the matrix h (Tn T12) g has the form (A 0), 
where A is a diagonal matrix. Consider the matrix 



rpl rpl 

- L 21 J 22 



h 0\ /T n T 1: 



1/ VT 21 T 22 



3n 3i2 
321 322 

A 

T21 gu + T 2 2 321 T21 .912 + T 2 2 322 

We see that the block T{ 2 is zero and the vector-row (T 21 T 22 ) is distributed 
as (T 2 i T 22 ). 

Thus the problem is reduced to the following form. Fix a matrix 



T = 



where x a , y r € K, and Ai, . . . , A n _i are real and nonnegative. Let fj,\, . . . , /i„ 
be the eigenvalues of the matrix TT* . We must find the image of the Lebesgue 
measure {dx\} . . . {dx n ^i}{dy{\ . . . {dy k } under the map 

(xi, . . .,x n -i;yi, ...,y k )^> (m, . . .,jj, n ). 



(\\' 2 








. 


. 


\ 





xT ■ 





. 


. 










.1/2 
• A n-l 


. 


. 




\ xi 


x 2 


Xn—1 


yi ■ 


• Vk 


) 



Now 









A 2 





.1/2 .1/2 

\\{ X\ A 2 ' X2 



\l/2- 

A/ x\ 

\l/2- 

A 2 x 2 



,1/2 _ 



A„-i 

a^-Wi EKI 2 + EM 2 7 



Define the new variables 



to 



The image of the measure {dx{\ . . . {dx n -i}{dyi} . . . {dx k } under the map 
(xi, . . .,x n -i;yi, . . . ,y k ) ^ (£1, . . . ,S, n -i,v), 

obviously is 



Jn-i)9 fee 



(6) T(k8) 



- i -n^ _i ^n«- 



fe=i 
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Now, the equation on the eigenvalues ji of the matrix TT* has the form 

n—l n—1 

X a - fj, 



a— 1 a— 1 

Let hi, . . . , Hn — be the roots of this equation; in other words we represent 
the left part of equation (4.6) in the form ftfe^] ■ Obviously, 

11 (Mp-Aa) 
A Q n (A/3 — A a )' 

= E^-E A «- ( 4 - 7 ) 

Lemma 4.1. T/ie following identity holds 

n 

n m p 

n A/? 

/3=1 



Proof. Consider the function 

U(x)-- 

Its residues are 



IIO ~ Mp) 

^no - Aa) ' 



res C/(.t) = -£ a ; res [/(x) = -fl-r^ ; res U(x) = /Up - V] A a . 



The sum of residues is 0. Comparing this with (4.8), we obtain (4.9). □ 
Thus, £ a and r\ are expressed in terms of (ii, ■ ■ ■ , Mp by formulas (4.7), (4.9). 
The Jacobian of the transformation (fii, ■ ■ ■ , M«) l— * (£ii • • • , £,n-i,v) can be eas- 
ily evaluated using the Cauchy determinant (2.19). As a result, we obtain 

II Og - Mp) 

d£i . . . d^n-idr} = — 1 ^ p<g ^" — ___rf Ml ... c ; Mn . (4.9) 

l^a^n-l l^a</3^n-l 

It remains to substitute (4.7), (4.9) and (4.10) to (4.5). 

4.3. Matrix integrals. Let m^n. Denote by B„ jTrl (K) the set of all 

matrices over K = R, C, H of size nx m with norm < 1; the term 'norm' means 
the norm of an operator from the Euclidean space K n to the Euclidean space 
K m . As above, let 6 = dimK/2. 

Integral (3.3) and Proposition 1.2 obviously imply 
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ra-1 



/It — J_ 
[] [detdTj^T}*)^-^ 1 - 6 det(l - {T}j{T}f) Tj ~ Ti+1 ~ 

B„, m (K) J=1 

x det(TT*) CT "~ (m ~" +1)e det(l - TT*) T ™~ 1 dT = 

= 7r mn8 TT r(g- j )r(r 3 ) 

11 r(aj + r 3 + (j - l)9)T((m j + 1)8) ' [ ' > 

Another partial case of integral (3.3) is (see Proposition 1.1) is the Gindikin 
beta-function (see |23], |T5 ] .|25 ] ) 



j Yl det[T]J^ CTj+1_e det(l - [T^) 7 ^'^" 



T=T*,0<T<1 i 

where the integration is given over the set of all the Hermitian matrices over K 
satisfying the condition < T < 1. 

The Whishart-Gindikin matrix gamma-function 



J ndet[T]^-^ +1 - e exp{-(^ -^ + i)tr[T],}: 



7 — 1 

Pos K (n) J 



x detT""- 1 exp{-^ntrT}di = ^ l[n - 1)9 \{Y{a )^ a ^ U ' 1)9 (4.12) 

i=i 

is a partial case of the integral (3.1). Another partial case of (3.1) is the integral 

„ n— 1 

J det({T} 3 {T}*yi-^- s exp{-(^ - ^ +1 ) tr{T},{T}*} x 

x det^T*)' 7 "-^"-" 14 - 1 ) exp{-V>„tr TT*}dT = 



Mat K (n,m) 3 1 



- CTj -(j-l)6 



= ^ff^ _ (4.13) 

11 r((m-j + l)(9) 1 J 

The Mehta type integral (3.3) interpolates the matrix integrals 

/ n— 1 
cx p{- o Z)^ ~ tr[T] ? ~ 9^" trT2 } dT - 

7 — 1 

Hcrm K (n) 

However these integrals themselves are quite obvious, since the expression in 
the exponent is a diagonal quadratic form. 



25 



References 

[1] Hankel H., Bestimmte Integrate mit Cylinder functionen. Math. Ann., 8 
(1876), 453-470. 

[2] Watson G.N. A treatise on the theory of Bessel functions. 2nd Ed., Cam- 
bridge Univ. Press, 1944 

[3] Olevskii M.N. On representation of arbitrary function as an integral with a 
kernel containing a hypergeometric function (Russian), Dokl. Akad. Nauk 
SSSR, 69, Nl, 11-14. 

[4] Koornwinder, T.H., Jacobi functions and analysis on noncompact symmet- 
ric spaces, in Special functions: group theoretical aspects and applications, 
eds. Askey, R., Koornwinder T.H., Schempp W., 1-85, D. Reidel Publ. Co., 
Dordrecht-Boston, 1984. 

[5] Neretin, Yu.A., Index hypergeometric transform and imitation of analysis 
of Berezin kernels on hyperbolic spaces, Mat. Sbornik , 192 (2001), No 
3,83-114; English transl.: Sbornik Math., 192:3, 403-432. 

[6] Heckman G.J., Opdam E.M. Root systems and hypergeometric functions I, 
Comp. Math.,, 64, (1987),, 329-352. 

[7] Heckman G.J., Schlichtkrull H. Harmonic analysis and special functions on 
symmetric spaces, Academic Press, 1994 

[8] MacDonald I.G. Symmetric functions and Hall polynomials. 2nd Ed., Ox- 
ford, Clarendon Press, 

[9] Cherednik I. Inverse Harish- Chandra transform and difference operators, 
Int. Math. Res. Notes, 15 (1997), 733-750. 

[10] Cherednik I., Ostrik V. From double affine Hecke algebra to Fourier trans- 
form, Preprint, available via http://arXiv.org/abs/math/0111130 

[11] Pickrell, D., Measures on infinite- dimensional Grassmann manifold. 
J. Funct. Anal., 70 (1987), 323-356. 

[12] Neretin, Yu.A., Hua type integrals over unitary groups and over inverse 
limits of unitary groups, Duke Math. J. 114 (2002), 239-266, Preprint 
http : //arXiv . org/abs/math/0010014 

[13] Borodin, A., Olshanski, G., Infinite random matrices and ergodic measures, 
Comm. Math. Phys., 223 (2001), 87-123. 

[14] Olshanski G. Gelfand-Tsetlin schemes and measures of hypergeometric 
type, to appear. 

[15] Gindikin, S.G., Analysis on homogeneous spaces (Russian), Uspehi mat. 
nauk,19, No 4, 3-92(1964); 



26 



[16] Neretin, Yu.A., Matrix analogs of B- function and Plancherel formula for 
Berezin kernel representations, Mat. Sbornik, 191 (2000), 5, 67-100 (Rus- 
sian); English transl.: Sbornik 191 (2000), 683-715 Preprint version is avail- 
able via |http : //arXiv . org/abs/math/9905045 

[17] Bhatia R. Matrix analysis, Springer, NY, 1997. 

[18] Gantmakher F.R. Lectures on analytic mechanics. Moscow, Nauka, 1966 

[19] Gelfand, I.M., Naimark M.I., Unitary representations of classical groups. 
Trudy Mat. Inst. Steklov.,v.36 (1950) (Russian); German translation: 
Gelfand I.M., Neumark M.A., Unitare Darstellungen der klassischen grup- 
pen., Akademie-Verlag, Berlin, 1957. 

[20] OkounkovA., Olshanski G. Shifted Jack polynomials, binomial formula, and 
applications, Math. Res. Lett., 4 (1997), 69-78. 

[21] Kazarnovsky-Krol A., Cycles for asymptotic solutions and the Weyl group, 
Gelfand mathematical seminars 93-95, Boston, Birkhauser, 123-150, 1996 

[22] Ingham A.E. An integral which occurs in statistics, Proc. Cambr. Phil. Soc, 
29 (1933), 271-276. 

[23] Siegel, C.L., Uber die analytische Theorie der quadratischen Formen, Ann. 
Math. 36 (1935), 527-606 

[24] Hua Loo Keng, Harmonic analysis of functions of several complex variables 
in classical domains. Beijing, 1958(Chinese); Russian transl.: Moscow, In- 
ostrannaya literatura (1959); English transl.: Amer. Math. Soc, Providence 
(1963) 

[25] Faraut, J., Koranyi, A., Analysis in symmetric cones. Oxford Univ. Press, 
(1994) 

[26] Neretin Yu.A., On separation of spectra in analysis of Berezin ker- 
nels Funkts. Anal. Prilozh.,34 (2000), 3 (Russian); English transl.: 
Funct. Anal. Appl. 34 (2000), 197-207. Preprint version is available via 
http : //arXiv . org/abs/math/9906075 

[27] Selberg A. Bemerkninger om et multipelt integral, Norske Mat. Tiddsskr., 
26, 71-78. 

[28] Kazarnovsky-Krol, A., Matrix elements of vertex operators of the deformed 
WA n algebras and the Harish- Chandra solutions to Macdonald's difference 
equations, Selecta Math., New ser., Basel, 5, 1-45, 1999. 

[29] Anderson G.W. A short proof of Selberg 's generalized beta-formula, Forum 
Math., 3, 415-417 

[30] Andrews, G.R., Askey, R., Roy, R., Special functions., Cambridge Univ. 
Press, 1999 



27 



Institute of the Theoretical and Experimental Physics, 
B. Cheremushkinskaya, 25 
Moscow 117259 
Russia 

neretinOmccme . ru 



28 



